Interactive Signal Processing for Acoustic Instruments

As high-quality signal processors become more sophisticated and programmable, composers are increasingly interested in using them to transform the sounds of acoustic instruments during a live performance. This paper describes a real-time interactive program, created with MAX, that allows a performer to generate and control sound from a MIDI-controllable signal processor according to musical cues and performance gestures. The program analyzes incoming performance data and responds by sending continuous controller messages to various parameters, giving the performer immediate feedback and the ability to alter the processed sound. MIDI signal processors have proliferated in the last few years, offering sophisticated algorithms that transform a sound in real-time. When used as integral parts of music compositions, they provide a common sound world for synthesized sounds and acoustic instruments to blend, while expanding the coloristic and expressive range of the instruments. Although these "effects boxes" use limited types of processes, their portability, price and programmability makes them attractive tools for use in concert performance. Increased flexibility is gained by using computers to send continuous controller messages, which allow for subtle, dynamic control over all the aspects of the processed sound. As the control of these processes becomes more flexible, it can also become more musical, by responding to a skilled musician, who controls how the parameters change over time. For this purpose, I created FollowPlay, a program written in MAX (Puckette, 1988) specifically designed for interactive composition. It allows musicians to generate the sound source for processing while sending performance cues to a computer, using performance gestures to specify exactly how the processing will take place, thereby controlling as many aspects of modified sound as the composer desires. The program is specifically designed to respond to acoustic instruments played via a microphone to a pitch-to-MIDI converter (which detects pitch and dynamics over time), but will respond equally as well to MIDI instruments. One aspect of the program is a collection of MAX patches, each designed to "interpret" incoming performance information. By looking at pitch, velocity, duration and time, the program is able to follow, analyze, store and compare various aspects of a performance. These are general tools for interactive composition (not just for interactive signal processing) and include modules that look at how the music is being played, detecting tempo, articulation, crescendo and diminuendo. Histograms store a specified number of events, capturing information about articulation, dynamics and phrasing, which allows the computer to look at whole sections of a piece. This enables the program to respond immediately to performance gestures, while also making decisions based on how a longer passages were played (Rowe, 1990). The signal processing is responsive to a particular performer or performance, empowering the musician with an extension of their sound sensitive to their musical nuance. Analytical methods to detect tempo and dynamics are quickly becoming the tools of the trade for interactive composition, and many people have already shown numerous useful algorithms that play music based on a live performance. While most of the algorithms for generating complex music are not useful signal processing, many of the techniques are. For example, sequences of continuous controller information can be triggered by the performer, creating complex timbral changes. The program uses the score-following features of the EXPLODE object (Puckette,